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Portfolios are central to the Bachelor of Science program in scientific and technical 
communication at one large state university in the midwestern United States. At the 
beginning of the program, students who want to gain admission must submit a portfolio 
that demonstrates both their ability to communicate in written form and an interest in the 
field. At the end of the program, students at some of the participating campuses (the 
program is offered in partnership between the main campus of the University and several 
of its sister campuses and other public institutions in the state) prepare an electronic 
portfolio that showcases their best work and that students use while searching for jobs. 

Throughout the program, the faculty supports students in the development of their 
portfolios. To help students prepare their initial portfolio, the University offers a 1-credit 
course for pre-majors that helps students learn what scientific and technical 
communication really is, then explains how to prepare the portfolio for admission. Students 
learn how to select items for the portfolio, how to describe those items to people who will 
be assessing their portfolios, and how to accurately represent their work. For example, 
students are encouraged to include works that they contributed to but not necessarily in a 
lead role, such as manuals for which they handled the layout and production, but not 
necessarily the writing. 

The portfolio serves as one of the primary pieces of evidence for acceptance into the 
program. During the program, students are reminded on a regular basis to add other 
pieces to their portfolios and prospective employers actively seek to view student work 
when interviewing students for jobs upon graduation. Although no one has publicly 
declared this portfolio program to be a success, that the academic program has used 
portfolios for so many years and that both students and faculty actively support it, the 
program can be considered one. 

At about the same time that the University was devising the portfolio program for its 
program in scientific and technical communication, the state's Board of Education tried to 
redesign the entire high school curriculum around the concept of portfolios. Readiness for 
graduation would involve a portfolio review. Despite support from the governor, influential 
state legislators, leading teachers, and the business community, the program was declared 




an unmitigated disaster almost as quickly as it was launched, and the state abandoned the 
program within a year. According to newspaper accounts at the time: 

Such terms as "invigilators," "performance packages," "rubric" and "authentic assessments" have been used to 
explain the state's ambitious overhaul of what students should know. 

The fact that many people may have no idea what they mean explains why the profile is in 
danger of flunking (Draper, 1999). 

As these two instances show, although portfolios have great potential as a tool for 
collecting and gauging long-term learning and the competencies that someone brings to a 
situation, their long-term success as a teaching and assessment tool in education is no 
sure thing. Nor should people assume that, because portfolios are widely used in some 
workplace situations that they can be used in all such situations. 

In this commentary, I reflect on the potentials and pitfalls of the e-portfolio concepts, 
using the other articles in this special issue as a springboard for discussion. Specifically, I 
explore issues associated with the literature that we consult when communicating about 
electronic portfolios, the conception of the roles of creators, facilitators and evaluators of 
the portfolio process, and the research on electronic portfolios. 

Reviewing the Literature: What's Here? What's Not? 

In reviewing this discussion about the use of portfolios, as important as it is to notice what 
is discussed in the review of the literature, it's probably more useful to notice what was 
left out. Two important issues regarding portfolios as a vital teaching and assessment tool 
are missing from these discussions: a substantive discussion of the use of paper portfolios 
as teaching and assessment tools, and a discussion of the use of portfolios beyond the 
formal educational system. 

Consider, first, the issue of paper portfolios. Abrami and Barrett (2005) note in their 
introduction to this special issue, 

EPs are the Information Age’s version of the artist’s portfolio in the sense that they not only summarize an artist’s 
creative achievements but also illustrate those achievements. An artist, architect, or engineer who displays her 
portfolio of work allows the viewer to form a direct impression of that work without having to rely on the judgments 
of others. EPs tell a story both literally and figuratively by keeping a temporal and structural record of events. 

Although most of the articles in this special issue document the ongoing conversation 
about the use of electronic portfolios, noticeably absent from most of these articles are 
research and other discussions about the use of traditional paper portfolios in individual 
courses, curricula, and workplace settings. At most, just a few citations are made. 
Although the "electronic" aspect of portfolios is relatively new (about the last ten years), 
the portfolio itself has had a much longer life. Indeed, a meeting at which experienced 
professionals review the portfolios of their colleagues is a popular type of monthly meeting 
for chapters of the American Institute of Graphic Arts (AIGA, 2001). 

Even though electronic portfolios significantly simplify the capture of samples, the 
provision of commentary, and transfer the portfolio materials among users over their paper 



counterparts, the general process of developing portfolios is not unique to the information 
age and, as Abrami and Barrett note, has a long history of use in the fine and applied arts. 
In academic programs in those fields, the use of portfolios has been integrated into the 
curriculum for decades, long before electronic portfolios were even feasible. Similarly, the 
use of portfolios in the job search process in those fields has been widely used for 
decades, too. As other disciplines consider ways to integrate portfolios into both the 
classroom and the job search process, we might spare ourselves some poor choices by 
learning from our colleagues' experience. 

As much of the conversation about the use of paper portfolios does not seem to be 
represented in this discussion, so is the discussion about the use of portfolios beyond the 
formal educational system. None of the articles discusses focuses on uses of electronic 
portfolios for work. That does not mean that such uses are not widely discussed in Canada. 
One key use of portfolios beyond the academy is as a tool for assessing prior learning. 
Such tools provide authentic assessments that help immigrants establish their professional 
credentials in a system that does not allow these people to transfer their certifications and 
licenses, and work in the fields for which they have been trained. 

Although most discussions of using portfolios beyond the academy focus on assessment 
and showcase uses, potential also exists in the use of process portfolios for the workplace. 
As careers become obsolete before workers reach the end of their careers, job placement 
specialists increasingly face the challenge of placing skilled workers in new careers. 
Because few mechanisms exist for helping workers leverage existing skills and gain 
advanced placement in new careers, most face the challenge of starting all over again, 
with no recognition of the skills or experience they have. Portfolios provide a means for 
assessing such skills and experience, and re-packaging them to workers' benefits. 

Preparing Electronic Portfolios: Determining the Roles and Responsibilities of 
Creators > Evaluators , and Facilitators 

Although presented as corollary discussions, most of the models and research presented in 
this issue eventually raise questions about the roles and responsibilities of creators, 
facilitators, and evaluators of electronic portfolios. One point of confusion pertains to the 
role and responsibility of portfolio creators in designing their electronic portfolios. On the 
one hand, Challis (2005) proposes that a mature user of electronic portfolios designs the 
navigation and visual design of the portfolio. Her model suggests that the navigation 
should be "clear and intuitive, let users select their own pathway(s), and fully hyperlinked" 
(2005). In terms of visual design, it should be "uncluttered and elegant, [use graphics] in 
accord with the portfolio's purpose and its creator, [avoid] distracting elements, [be] well 
organized and coherent, and [have] connections [that] are readily made" (2005). This 
sounds like advice that a professor offers to students who are majoring in graphic design 
and professional communication (Kostelnick & Roberts, 1998); is this reasonable to expect 
of students and professionals working in other fields? 

On the other hand, emphasizing the visual aspects of the electronic portfolio often pre¬ 
empts attention to its substance, as one of the students in Tosh, Light, Fleming, and 



Haywood's (2005) notes. The resolution is not simple. Students in the same study 
expressed concern that template-based tools, which seem to be the dominant form of 
software for electronic portfolios, lacked the flexibility students felt they needed to 
accurately represent their work. 

In the end, the question of how much creators should be involved in designing the 
navigation and appearance of their electronic portfolios is unresolved. Perhaps software for 
electronic portfolios could be designed to allow for more flexibility, learning a lesson from 
the layered user assistance provided for other types of software. Layered assistance 
provides people with increasing levels of flexibility and freedom as they reach more 
experienced levels of use (Mobley, Knight, & Meserth, 2003). For example, electronic 
portfolio software might provide novice users with structured templates for preparing their 
materials while letting experienced users bypass the templates. It seems that few systems 
are designed for multiple levels of users. 

If the roles of creators are not well-defined, neither are those facilitators and evaluators. 
Facilitators are the people who instruct people on how to create electronic portfolios and 
support these people in creating their portfolios. The nature of the instruction and advice 
they offer is likely to change at various phases of a creator's academic and professional 
careers. For example, the guidance needed to help a seven-year-old prepare a process 
portfolio substantially differs from the guidance needed to help a 47-year-old prepare a 
showcase portfolio for a creative director's job. 

The role of the evaluator is similarly unclear. Immediate issues facing evaluators of 
electronic portfolios include acceptance and evaluation rubrics. Consider the challenges of 
acceptance. In certain situations when electronic portfolios might make excellent sense, 
evaluators do not seem comfortable with electronic portfolios. For example, promotion and 
tenure—for which candidates often need to provide 2 three-inch binders of documentation 
—seems like a natural candidate for electronic portfolios. Instead of placing these 
oversized binders in an inconvenient but centralized location for committee members to 
read, candidates can place provide committee members with individual, portable CDs and 
DVDs, which can be viewed anywhere. But as ChaMis (2005) reports, committee members' 
first question is, "How do I print this?" suggesting that they may not yet be ready for 
electronic portfolios. 

Similarly, once evaluators receive portfolios, they often receive little or no guidance in 
evaluating them. This is not a promotion and tenure issue; this seems to be an ongoing 
issue with all types of portfolios. For example, although all of the articles in this issue 
consider evaluation, none provides an example of the specific rubrics for evaluating a 
portfolio. In fact, one of the concerns expressed by students in the Tosh, Light, Fleming, 
and Haywood (2005) article is that the assessment criteria are not provided to them, 
especially for such subjective assignments as writing personal reflections. 

What these articles do suggest is that evaluation must be a tedious task, because several 
authors mention the amount of time that evaluation of an individual portfolio takes, and 



this may place a practical limit on the extent to which portfolios may be used. 

But what do evaluation rubrics actually look like? Perhaps the rubrics for evaluating the 
portfolios of senior (experienced) practitioners who apply to the Certified Training and 
Development Professional program offered by the Canadian Society for Training and 
Development provide an insight into what evaluation criteria might look like. 


Knowledge 

Points 

Degree in a related field e g.: 

Organizational development Adult education 

Human resource mgmt 

Educational organizational psychology 

PhD 20 

Masters 10 

BA BEd 5 

Certificates or Diploma in related topics e g.: 

• OSTD Advanced PDAP 

• CSTD Approved programs at: 
o Rverson 

o OISE 

o Durham 

• Others (that may be considered) 
o St Francis Xavier 

o ASTD i.e. HPI Certificate 

5 

Teaching Experience in T&D program at a recognized 
college university 

5 

Experience 

Points 

Years of experience in T&D or OD 

25 yrs-20 

20 vrs-15 

15 vrs-10 

10vrs-5 

Manager with greater than 3 years in the position 

• responsible for T&D function in a medium to large size 
organization 

• and responsible for the coaching development of trainers 

10 points 

Skills Demonstration 

Points 

Award runner in relevant area e g.: 

• President's award. 

• OTTER award i.e .: principal contributor 

10 points 

Published author on related topic 

10 points 

Lead Researcher and or development of best 

practices in T & Y). Research must be published, presented, or 

credited in a publication or a public presentation. 

10 points 

Noted public speaker or expert panelist witness for major 
organizations or conferences on related topics (a minimum of 
three separate occasions) 

10 points 

Leadership in T& D through CSTD Board or Chapter 
involvement as 

• President 

• Planning Committee 

• Conference Committee 

Chair 10 

Committee 

Member 5 


















• Awards Committee 

{Or involvement in a related organization) 


Recognized products andorT&D services that meet or exceed 

10 

the competencies outlined by the Training Competency 


Architecture 


References 


Sugge st a minimum of 2 - 3 people who can validate your abilities, and whom we can 

contact for this purpose. Please list their name, title, place of work, phone number, and 

e-mail. 



Figure 1 : Criteria for Evaluating a Senior Practitioner's Portfolio for the Canadian Society for Training and 
Development's certification of Training and Development Professionals 

But these are only rubrics for evaluating an assessment portfolio of an experienced 
professional. These evaluation criteria are prescriptive, representing what experts believe 
are the competencies of a senior training and development practitioner, rather than ones 
emerging directly from observation of the work in the field. These evaluation criteria are 
also rooted in a behaviourist approach, focusing primarily on accomplishments rather than 
the processes followed to achieve them. This behaviourist approach sharply contrasts with 
the socio-constructivist approach advocated by Abrami and Barrett (2005) in their 
introduction and Wade, Abrami, and Sclater (2005), and alluded to by the other articles in 
this special issue. 

This does not mean that the socio-constructivist paradigm is incompatible or unworkable 
with electronic portfolios; indeed that is not my point in raising it. Indeed, the discourse 
analysis of the electronic portfolios prepared by first-year pre-service teachers reported by 
Peters, Chevrier, Leblanc, Fortin and Malette (2005) suggests a way to approach the 
evaluation of electronic portfolios from a descriptive framework. 

The point merely suggests that electronic portfolios are ultimately compatible with a wide 
variety of teaching and evaluation philosophies. The crux of that evaluation is in the 
rubrics and, without samples of those rubrics, facilitators and evaluators will feel 
challenged in applying electronic portfolios in constructivist and other contexts. 

Reviewing the Research: When to Look Inward, When to Look Outward 

As the roles of the various participants in the portfolio process need further clarification, so 
do issues raised by the research conducted thus far. Specifically, three issues arise when 
considering the research presented in this issue. First, all of the studies reported here 
focus on the use of e-portfolios in academic contexts. Some, like Wade, et al. (2005), 
focus on the use of e-portfolios in schools, while more, like Challis (2005), Milman and 
Kilbane (2005), Smits, et al. (2005), and Tosh, et al. (2005) all look at the use of 
electronic portfolios in higher education. Indeed, three of the contributions—those by 
Milman and Kilbane, Peters, et al., and Smits, et al.—not only report on the use of e- 
portfolios in teacher education, but within the context of their own classrooms and 
programs. 

Although the lack of reports on the use of e-portfolios in workplace contexts is a source of 








concern, evidence in the literature suggests that they do exist (e.g., Dagley & Berrington, 
2005; Sanders, 2005). 

What is more troublesome, however, is the extensive use of our own classrooms to 
conduct research on e-portfolios, the second issue that arises when considering the 
research reported in this issue. Although conducting research in one's own classroom is 
widely used in educational research because of its convenience, it often poses more 
problems in reporting than are offset by the convenience. Milman and Kilbane address 
some key methodological issues in their article. But they overlook one key issue. One of 
the goals of researching e-portfolios is to determine the contexts in which electronic 
portfolios "are most effective and worthwhile" (Abrami & Barrett, 2005). But if we limit the 
contexts in which we conduct the research and look only at our own classrooms (and those 
of our colleagues in education programs), we are in no position to make such an 
assessment. For research on contexts to be effective, researchers need to look beyond our 
own classrooms in teacher education. Furthermore, other teaching disciplines, like writing 
centers, are experimenting with the use of electronic portfolios (Click & Magruder, 2004) 
and dental education, often without the involvement of researchers in the mainstream of 
education. Perhaps, in seeking classrooms to study, we should focus on developing 
partnerships with these other disciplines. 

But perhaps e-portfolios are not yet ready for extensive summative research, as 
suggested by the third, and probably most significant, issue that arises when considering 
the research in this issue. As Tosh, et al.'s (2005) report of student perceptions of pilot 
implementations of electronic portfolios in higher education programs suggests, perhaps 
efforts to conduct summative evaluation of the outcomes and processes of electronic 
portfolios and evidence of their success, as well as the related concern about moving 
forward with funding and infrastructure (Abrami & Barrett, 2005) are premature. With 
methods for developing, teaching with, and evaluating electronic portfolios still in flux, 
perhaps the current research agenda should take a more modest approach: focusing on 
formative issues, such as validating teaching techniques, instructions to students, and 
evaluation rubrics. As most basic instructional design texts advise, one should only 
conduct summative evaluation on validated materials (Dick, Carey, & Carey 2000; Smith & 
Ragan, 1999). The evidence presented by Tosh, et al. suggests that, perhaps, e-portfolios 
are not yet validated; they are still in the formative stage. In addition to validating the 
educational aspects of e-portfolios, formative evaluation could address the technical and 
administrative issues (Abrami & Barrett, 2005) raised by the use of e-portfolios. 

Concluding Thoughts: Thinking Broadly, But Acting Specifically 

Admittedly, until now, this commentary has focused on the shortcomings in the literature 
reviews, designs, and research on electronic portfolios. That's because the initial 
enthusiasm expressed for electronic portfolios resembles the initial enthusiasm for e- 
learning, which was projected to cause an immediate revolution in all forms of education 
(Milken 1998). Although some research and theory supported this enthusiasm, they 
ignored other research (such as Russell, ongoing) and the practicalities of designing and 



implementing e-learning (Van Buren & Sloman, 2003), and these ultimately meant that e- 
learning launched an evolution of education that continues to this day. Although I strongly 
believe in the potential of electronic portfolios, I would encourage those of us who 
advocate for their use to take a more modest approach, tempered by the lessons of e- 
learning as well as the evidence in our own research and theory development. Although 
electronic portfolios have much potential to benefit the learning and working processes, 
the evidence seems to suggest that they are only at a formative evaluation stage of 
development. So perhaps some of the theory development, research, and product 
development—as well as rhetoric surrounding them—should be scaled to something more 
appropriate to this phase of development. 

In terms of theory development, while the concept of portfolios for life makes tremendous 
sense because portfolios have can be used in all phases of education and career 
development, perhaps the concept of one portfolio for life might be premature. As the 
research and commentary published elsewhere in this issue suggest, we have much to 
learn about the use of portfolios within each specific school context and even more to learn 
about their use within work contexts. In addition to having strong evidence of the value of 
electronic portfolios in these contexts, exploring the concept of a portfolio for life requires 
studying the use of portfolios in passages between major phases of schooling and careers. 
For example, how would a portfolio prepared in elementary school need to be modified for 
use in middle school? From middle school to high school? If we do not have validated 
portfolios for elementary and middle schools, how can we explore the transition of 
portfolios between the two? 

Similarly, some technical issues arise when considering the concept of a portfolio for life. 
Although many people first raise the issue of standards, a more immediate issue is the 
likelihood that file formats will change several times over the course of a real person's 
lifetime. The dominant file formats used now—such as .doc, .ppt, .jpg, and so on—are 
barely fifteen years old. Although their dominance limits software publishers' flexibility in 
changing file formats, changes in technology might demand that such changes happen. 
Given the rate at which technology changes, a strong likelihood exists that, even if 
standards for a single portfolio for life could be developed, the practical realities of 
technology will require that creators of portfolios convert their electronic artefacts at least 
once during their lifetime. That, in turn, may require that the creator discard the previous 
portfolio and that software developers abandon any standards developed. 

The discussion of the current limits of the portfolio-for-life concept also suggest research 
activities that should occur at this time. First this emerging discipline needs more 
formative research, like that conducted by Tosh, et al., to validate that the electronic 
portfolios can actually achieve the goals for which they were intended. As mentioned 
earlier, this research needs to be conducted not only in our own classrooms in education 
programs but, more significantly, in other programs and in the workplace. Only when 
electronic portfolios are validated should summative evaluation commence. And only after 
these portfolios are evaluated in the contexts for which they were designed should 



researchers consider linking portfolios used in one context with those used in another. 


That, in turn, suggests the focus of product development for electronic portfolios. Although 
the concept of electronic portfolios for life has long-term appeal, in the short-term, 
developing software for specific contexts might make more sense. More significant than 
developing software for specific contexts is the development of related materials for use 
with this software. Creators of electronic portfolios need guidance on how to create these 
materials, and support and guidance as they develop their nascent portfolios. Facilitators 
need resources and training in how to teach people to create their portfolios and how to 
integrate the portfolios into their teaching and work contexts. Evaluators of portfolios need 
criteria for assessing portfolios, not only to assess existing portfolios, but models of 
assessments so rubrics for other types of portfolios can be developed. 

The last challenge is neither a technical nor a research challenge, but one that was 
introduced at the beginning of this commentary, suggested in the section on evaluation, 
and is well-known to most people who introduce software into an organization: managing 
the change caused by the introduction of this new approach. Evidence suggests that 
electronic portfolios have the potential to promote self-regulation in learners, provide more 
authentic assessment of learning activities, demonstrate competence for jobs, and keep 
an ongoing record of accomplishments and learning. But if those of us who are proponents 
of electronic portfolios mismanage their introductions in our contexts as badly as the state 
department of education described in the beginning of this article introduced portfolios in 
its context, we won't have the opportunity to demonstrate that. 

Fortunately, even without the research community, a case for electronic portfolios is 
emerging if, for no other reason, many people are already creating their own electronic 
portfolios, many without the software tools that we are creating. For example, visit the 
web sites of various independent professionals, who proudly display their portfolios as 
evidence of their worthiness for work, like marketing writer Jack Massa does at 
www.guidancecom.com and communications and training firm, Fredrickson 
Communications (www.fredcomm.com). These people did not create their portfolios as a 
requirement for class or as part of a research project; they did so out of necessity. 

Ultimately, our ability to integrate electronic portfolios into schools and the workplace 
rests not with our exuberance for the software, but with our ability to ensure and 
communicate the viability, practicality, and utility of these tools. One of the best ways to 
do is by taking the suggestion of the students in Tosh, et al.'s study: create electronic 
portfolios of our own work. Perhaps that's the best way to begin introducing electronic 
portfolios. 
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